Skip to content

Conversation

@tobi
Copy link
Member

@tobi tobi commented Jan 28, 2026

Summary

This PR introduces a native C parser and bytecode compiler for Liquid control flow tags, providing significant performance improvements.

Performance Improvements

Template Type Speedup
Variable-heavy parsing ~10x
Filter-heavy parsing ~4-5x
Control flow parsing ~2-4x
if/case rendering ~2x

New Components

  • Arena Allocator (arena.c/h): Fast bump-pointer allocation
  • AST Structures (ast.c/h): 21 node types for all Liquid constructs
  • Template Parser (template_parser.c/h): Recursive descent parser
  • Code Generator (codegen.c/h): AST → bytecode compiler

New VM Opcodes (15+)

  • Control flow: OP_JUMP, OP_JUMP_IF_FALSE, OP_JUMP_IF_TRUE
  • Comparisons: OP_CMP_EQ/NE/LT/GT/LE/GE/CONTAINS
  • Variables: OP_ASSIGN, OP_INCREMENT, OP_DECREMENT
  • Loops: OP_FOR_INIT, OP_FOR_NEXT, OP_BREAK, OP_CONTINUE

Native vs Ruby

Native C Still Ruby
if/elsif/else/unless Custom tags
case/when Filters
assign/increment/decrement render/include
Comparisons Drops

Bug Fixes

  • blank keyword comparison
  • empty keyword comparison
  • nil contains check
  • ✅ Bus Error crash in case/when
  • ✅ break/continue in for loops

Test plan

  • All 272 unit tests pass
  • 187 new tests for parser/VM
  • 91.5% liquid-spec conformance
  • Benchmark on production templates

tobi added 5 commits January 27, 2026 23:11
This PR introduces a native C parser and bytecode compiler for Liquid
control flow tags, providing significant performance improvements for
template parsing and rendering.

## Performance Improvements

- Parsing: 2-10x faster depending on template complexity
- Control flow rendering: ~2x faster for if/case statements
- Variable-heavy templates: up to 10x faster parsing

## Architecture

### New Components

- **Arena Allocator** (arena.c/h): Fast bump-pointer allocation for AST nodes
- **AST Structures** (ast.c/h): 21 node types for all Liquid constructs
- **Template Parser** (template_parser.c/h): Recursive descent parser
- **Code Generator** (codegen.c/h): AST to bytecode compiler

### New VM Opcodes

Control flow:
- OP_JUMP, OP_JUMP_IF_FALSE, OP_JUMP_IF_TRUE (with wide variants)

Comparisons:
- OP_CMP_EQ, OP_CMP_NE, OP_CMP_LT, OP_CMP_GT, OP_CMP_LE, OP_CMP_GE
- OP_CMP_CONTAINS

Variables:
- OP_ASSIGN, OP_INCREMENT, OP_DECREMENT

Loop control:
- OP_FOR_INIT, OP_FOR_NEXT, OP_FOR_CLEANUP
- OP_BREAK, OP_CONTINUE

### What's Native vs Ruby

**Native C execution:**
- if/elsif/else/unless - native jump opcodes
- case/when/else - native comparison + jumps
- assign - native OP_ASSIGN
- increment/decrement - native opcodes
- All comparison operators

**Still uses Ruby (by design):**
- Custom tags (extensibility)
- Filters (Ruby filter methods)
- render/include tags
- Drops and complex objects

## Bug Fixes

- Fixed blank keyword comparison (x == blank)
- Fixed empty keyword comparison (x == empty)
- Fixed nil contains check (array contains nil)
- Fixed Bus Error crash in case/when with multiple branches
- Fixed break/continue in for loops

## Tests

- 272 unit tests passing
- 187 new tests for parser/VM functionality
- 91.5% liquid-spec conformance (108/118)
str_forloop was declared but never assigned the "forloop" string,
causing OP_FOR_INIT/NEXT/CLEANUP to use Qnil as a hash key instead
of "forloop" when managing forloop state in context scope.

This could cause undefined behavior when:
- Running native for loops
- Mixing C and Ruby templates with for loops
- GC occurring during for loop execution
- Update Liquid gem to 5.11.0 which adds Liquid::Utils.to_s method
- Update parse_expression to accept new safe: keyword argument
- Fix adapter loading order to define Liquid::Environment shim early
- Fix strict_errors vs strict_variables confusion in render block
- Use static_environments instead of environments for global data

All 710 liquid-spec tests now pass with max complexity 400/400.
…perties

Changes:
- Fix null/nil comparisons to return false instead of raising ArgumentError
- Add to_liquid_value unwrapping for drop comparisons and truthiness checks
- Add string property access for .first and .last (returns first/last char)
- Add to_liquid method to Blank and Empty classes

Results: 4060/4102 tests passing (98.98%)
- Basics: 710/710 (100%)
- Liquid Ruby: 1518/1543 (98.4%)
- Shopify Production Recordings: 1832/1849 (99.1%)

Remaining 42 failures are mostly Shopify-specific case statement quirks
(multiple else clauses, when after else, fall-through) that would require
significant parser changes to support without breaking normal behavior.
Implements Shopify's quirky case statement behavior:
- When tags after else are now allowed
- Multiple else clauses all execute (until a when matches)
- Multiple matching when clauses all execute (fall-through)

Results: 4067/4102 tests passing (99.15%)
- Basics: 710/710 (100%)
- Liquid Ruby: 1521/1543 (98.6%)
- Shopify Production Recordings: 1836/1849 (99.3%)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant